Adaptive Streaming Perception using Deep Reinforcement Learning

Executing computer vision models on streaming visual data, or streaming perception is an emerging problem, with applications in self-driving, embodied agents, and augmented/virtual reality. The development of such systems is largely governed by the accuracy and latency of the processing pipeline. While past work has proposed numerous approximate execution frameworks, their decision functions solely focus on optimizing latency, accuracy, or energy, etc. This results in sub-optimum decisions, affecting the overall system performance. We argue that the streaming perception systems should holistically maximize the overall system performance (i.e., considering both accuracy and latency simultaneously). To this end, we describe a new approach based on deep reinforcement learning to learn these tradeoffs at runtime for streaming perception. This tradeoff optimization is formulated as a novel deep contextual bandit problem and we design a new reward function that holistically integrates latency and accuracy into a single metric. We show that our agent can learn a competitive policy across multiple decision dimensions, which outperforms state-of-the-art policies on public datasets.Comment: 19 pages, 17 figure

Too Brittle To Touch: Comparing the Stability of Quantization and Distillation Towards Developing Lightweight Low-Resource MT Models

Leveraging shared learning through Massively Multilingual Models, state-of-the-art machine translation models are often able to adapt to the paucity of data for low-resource languages. However, this performance comes at the cost of significantly bloated models which are not practically deployable. Knowledge Distillation is one popular technique to develop competitive, lightweight models: In this work, we first evaluate its use to compress MT models focusing on languages with extremely limited training data. Through our analysis across 8 languages, we find that the variance in the performance of the distilled models due to their dependence on priors including the amount of synthetic data used for distillation, the student architecture, training hyperparameters and confidence of the teacher models, makes distillation a brittle compression mechanism. To mitigate this, we explore the use of post-training quantization for the compression of these models. Here, we find that while distillation provides gains across some low-resource languages, quantization provides more consistent performance trends for the entire range of languages, especially the lowest-resource languages in our target set.Comment: 16 Pages, 7 Figures, Accepted to WMT 2022 (Research Track

Breaking Language Barriers with a LEAP: Learning Strategies for Polyglot LLMs

Large language models (LLMs) are at the forefront of transforming numerous domains globally. However, their inclusivity and effectiveness remain limited for non-Latin scripts and low-resource languages. This paper tackles the imperative challenge of enhancing the multilingual performance of LLMs, specifically focusing on Generative models. Through systematic investigation and evaluation of diverse languages using popular question-answering (QA) datasets, we present novel techniques that unlock the true potential of LLMs in a polyglot landscape. Our approach encompasses three key strategies that yield remarkable improvements in multilingual proficiency. First, by meticulously optimizing prompts tailored for polyglot LLMs, we unlock their latent capabilities, resulting in substantial performance boosts across languages. Second, we introduce a new hybrid approach that synergizes GPT generation with multilingual embeddings and achieves significant multilingual performance improvement on critical tasks like QA and retrieval. Finally, to further propel the performance of polyglot LLMs, we introduce a novel learning algorithm that dynamically selects the optimal prompt strategy, LLM model, and embeddings per query. This dynamic adaptation maximizes the efficacy of LLMs across languages, outperforming best static and random strategies. Our results show substantial advancements in multilingual understanding and generation across a diverse range of languages

Consumer Segmentation and Knowledge Extraction from Smart Meter and Survey Data

Many electricity suppliers around the world are deploying smart meters to gather fine-grained spatiotemporal consumption data and to effectively manage the collective demand of their consumer base. In this paper, we introduce a structured framework and a discriminative index that can be used to segment the consumption data along multiple contextual dimensions such as locations, communities, seasons, weather patterns, holidays, etc. The generated segments can enable various higher-level applications such as usagespecific tariff structures, theft detection, consumer-specific demand response programs, etc. Our framework is also able to track consumers' behavioral changes, evaluate different temporal aggregations, and identify main characteristics which define a cluster

Evaluating Demand Response Programs: Getting the Key Performance Indicators Right

Demand Response (DR) has recently garnered great attention, with many DR programs being deployed and evaluated worldwide. They are hailed as a significant benefit enabled by the Smart Grid and an efficient method to engage consumers in managing their energy usage and reduce environmental impact and costs. But while the opportunities are great, challenges still remain to exploit the untapped potential of DR. Due to the many diverse technological and social contexts where is applied, establishing a common framework for evaluating DR programs is a rather complex but essential task in order to design more efficient and easily adopted, by utilities and users, DR programs. In this paper, we apply in practice some of, already defined in literature, Key Performance Indicators, aiming to evaluate different DR programs and we assess their applicability. In that context, we present and discuss initial results from two indicative trial sites (residential and commercial) and provide suggestions for future DR designers. Finally, we introduce the DR dashboard, a way to get an overview of a DR system and visualize the indices calculated in each trial site

iDR: Consumer and Grid Friendly Demand Response System

Peak demand is a major challenge for power utilities across the world. Demand Response (DR) is considered to be effective in addressing peak demand by altering consumption of end consumers, so as to match supply capability. However, an efficient DR system needs to respect end consumer convenience and understand their propensity of participating in a particular DR event, while altering the consumer demand. Understanding such preferences is non-trivial due to the large-scale and variability of consumers and the infrastructure changes required for collecting essential (smart meter and/or appliance specific) data. In this paper, we propose an inclusive DR system, iDR, that helps an electricity provider to design an effective demand response event by analyzing its consumers’ house-level consumption (smart meter) data and external context (weather conditions, seasonality etc.) data. iDR combines analytics and optimization to determine optimal power consumption schedules that satisfy an electricity provider’s DR objectives - such as reduction in peak load - while minimizing the inconvenience caused to consumers associated with alteration in their consumption patterns. iDR uses a novel context-specific approach for determining end consumer baseline consumptions and user convenience models. Using these consumer specific models and past DR experience, iDR optimization engine identifies -(i) when to execute a DR event, (ii) who are the consumers to be targeted for the DR, and (iii) what signals to be sent. Some of iDR’s capabilities are demonstrated using real-world house-level as well as appliance-level data

MEGA: Multilingual Evaluation of Generative AI

Generative AI models have shown impressive performance on many Natural Language Processing tasks such as language understanding, reasoning, and language generation. An important question being asked by the AI community today is about the capabilities and limits of these models, and it is clear that evaluating generative AI is very challenging. Most studies on generative LLMs have been restricted to English and it is unclear how capable these models are at understanding and generating text in other languages. We present the first comprehensive benchmarking of generative LLMs - MEGA, which evaluates models on standard NLP benchmarks, covering 16 NLP datasets across 70 typologically diverse languages. We compare the performance of generative LLMs including Chat-GPT and GPT-4 to State of the Art (SOTA) non-autoregressive models on these tasks to determine how well generative models perform compared to the previous generation of LLMs. We present a thorough analysis of the performance of models across languages and tasks and discuss challenges in improving the performance of generative LLMs on low-resource languages. We create a framework for evaluating generative LLMs in the multilingual setting and provide directions for future progress in the field.Comment: EMNLP 202

DRSim: A Cyber Physical Simulator for Demand Response Systems

Demand Response (DR) is a mechanism in which electricity consumers alter their demand in response to power grid’s supply and economic conditions. DR programs have the potential to improve resource efficiency, sustainability, grid reliability and economic viability by providing tighter alignment between demand and supply. However, implementing DR program is a non-trivial task as it requires good knowledge of electricity consumption preferences, economic models and contextual factors. Developing such knowledge through real world studies can be expensive and be time consuming. As a result, utility companies have been finding it complicated to analyze potential viability and return on investments of DR programs for various ‘what-if’ scenarios. To address this problem, we present DRSim – a cyber-physical simulator that allows utility companies to study demand side participation aspects of communities with various practical scenarios. DRSim is based on the principles of agent-oriented modeling of users’ behavior and context. It is able to model the emergent behavior of a community based on real data traces that contain partial information about the environment. DRSim is a highly extensible framework to accommodate new data sources, new analytical functionalities and evolve its modeling power. Feasibility experiments show the modeling and analysis potential of DRSim in practical settings

Retrieval Augmented Chest X-Ray Report Generation using OpenAI GPT models

We propose Retrieval Augmented Generation (RAG) as an approach for automated radiology report writing that leverages multimodally aligned embeddings from a contrastively pretrained vision language model for retrieval of relevant candidate radiology text for an input radiology image and a general domain generative model like OpenAI text-davinci-003, gpt-3.5-turbo and gpt-4 for report generation using the relevant radiology text retrieved. This approach keeps hallucinated generations under check and provides capabilities to generate report content in the format we desire leveraging the instruction following capabilities of these generative models. Our approach achieves better clinical metrics with a BERTScore of 0.2865 ({\Delta}+ 25.88%) and Semb score of 0.4026 ({\Delta}+ 6.31%). Our approach can be broadly relevant for different clinical settings as it allows to augment the automated radiology report generation process with content relevant for that setting while also having the ability to inject user intents and requirements in the prompts as part of the report generation process to modulate the content and format of the generated reports as applicable for that clinical setting